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DETAILED ACTION 
Response to Amendment 

1. The amendments filed on 06/06/2003 have been entered. Claims 1, 4-11, 13, and 15-18 
have been amended. Claims 2, 12, 19, 20 have been canceled. Claims 21-28 have been newly 
added. 

Claim Rejections - 35 USC § 112 

2. The following is a quotation of the first paragraph of 35 U.S.C 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

3 . The amended Claim 1 is rejected under 35 U.S.C. 112, first paragraph, as failing to 
comply with the written description requirement. The claim(s) contains subject matter which 
was not described in the specification in such a way as to reasonably convey to one skilled in the 
relevant art that the inventor(s), at the time the application was filed, had possession of the 
claimed invention. The terms such as "an analysis module", "the calibration window", "the 
adjacent test windows" shown in the claim are not found in the specification. In addition, the 
following claim limitation shown in claim 1 is not supported by the specification, i.e., "an 
analysis module to determine analysis data for pixels within the calibration window, based on 
HSV data array, and to determine test analysis data for a set of adjacent test windows, each of the 
adjacent test windows having a same shape as a calibration window, where tracking data, to 
track the selected object, is selected from one of the calibration window and the adjacent test 
windows having a highest tracking probability." 
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4. Claims 3-8, 21-24 are rejected the same way as claim 1 because they do not correct the 
dependency on claim 1 . 

5. The amended Claim 9 is rejected the same way as claim 1 because the claim 9 recites 
similar terms and claim limitation as set forth in claim 1 . 

6. Claims 10-11, 13-18, 25-28 are rejected the same way as claim 9 because they do not 
correct the dependency on claim 9. 

Claim Rejections - 35 USC§ 102 

7. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application tor 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
kiternational application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in tiie United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

8. Claims 1,3-11, 13-18, 20-28 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Bradski et al. U.S. Pat. No. 6,363,160 (hereinafter Bradski). 

9. Claim 1: 

Bradski teaches an automated calibration system (figure 1; column 3, lines 34-55) to 
track a selected object through a series of frames of data (column 7, lines 55-65), comprising: 

(a) A display device (e.g., figure 1 1) to display at least one image frame received from 
an image input device, wherein the image frame includes a calibration window (column 4, lines 
2-60; column 15, lines 13-58; figures 4, 10A and 10B); 
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(b) An image selection device to select, via the calibration window, the selected object in 
the at least one image frame (column 3, lines 59-67, column 4, lines 1-10, and column 7, lines 

33-54; column 15, lines 13-58); 

(c) An image source device (video source 1 102 of figure 1 1) to provide a hue saturation 
value (HSV) data array of pixels forming the at least one image frame (column 4, lines 2-57; 
column 13, lines 45-52; column 15, lines 13-58; figures 3A and 3B); and 

(d) An analysis module (the flow diagrams of figures 5, 7 and 9; column 13, lines 55-67; 
column 14, lines 1-59) to determine analysis data for pixels (column 4, lines 2-57; figures 4, 3 A 
and 3B) within the calibration window (column 5, lines 52-67; column 6, lines 1-44; column 15, 
lines 13-58; figures 4, 10A and 10B), based on the HSV data array (column 4, lines 2-57; figures 
3A and 3B), and determine test analysis data (e.g., window sizing parameters) for a set of 
adjacent test windows (e.g., search windows or boxes or windows of FIG. 14A displaying 
images; column 5, lines 52-67; column 6, lines 1-44; column 15, lines 13-58), each of the 
adjacent test windows (e.g., the moving search windows) having a same shape as the calibration 
window (e.g., the entire video frame or a calculation region or a search window; column 7, lines 
5-55; column 15, lines 13-58), wherein tracking data, to track the selected object (e.g., the 
selected gestures), is selected from one of the calibration window (e.g., the calculation region) 
and the adjacent test windows (e.g., the search windows) having a highest tracking probability 
(i.e., the tracking data such as window location parameters are determined/adjusted and a search 
window having the largest connected region of a probability distribution and the greatest 
probability density is selected, column 6, lines 45-67; column 7, lines 55-67; column 8, lines 1-9; 
column 15, lines 13-58). 
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Claim 3: 

The claim 3 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a digital camera. However, Bradski further discloses the claimed limitation 
of the digital camera (column 4, lines 1-57). 

Claim 4: 

The claim 4 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of a single device. However, Bradski further discloses the claimed limitation 
of the single device (figure 1). 

Claim 5: 

The claim 5 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the analysis module calculating a mean hue and a standard deviation of a 
hue of the pixels representing the selected object. However, Bradski further discloses the claimed 
limitation of the analysis module (figures 5, 7 and 9) calculating a mean hue and a standard 
deviation of a hue of the pixels representing the colored object (figures 3A, 3B and 4; column 4, 
lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 12, 

lines 1-67). 

Claim 6: 

The claim 6 encompasses the same scope of invention as that of claim 5 except additional 
claimed limitation of the selected object not being tracked if the mean hue or the standard 
deviation of the hue is less than predetermined levels. However, Bradski further discloses the 
claimed limitation of the colored object not being tracked if the mean hue or the standard 
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deviation of the hue is less than predetermined levels (It is apparent that a Gaussian or Chi- 
Square hue probability distribution of Bradski involves the mean hue or the standard deviation of 
the hue and search for the area with the greatest probability density involves the motion-tracking 
processing of not tracking a region where the mean hue or the standard deviation of the hue 
being less than predetermined levels; figures 3 A, 3B and 4; column 4, lines 2-57, column 10, 
lines 1-16, column 5, lines 1-63, and column 6, lines 22-44; column 8, lines 10-38; column 12, 
lines 1-67). 

Claim 7: 

The claim 7 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the analysis module calculating a mean saturation and a standard deviation 
of a saturation of the pixels representing the selected object. However, Bradski further discloses 
the claimed limitation of the analysis module (e.g., figures 5, 7, and 9) calculating a mean 
saturation and a standard deviation of a saturation of the pixels (e.g., a mean saturation and a 
standard deviation of a saturation of the pixels is an inherent property of a Gaussian or Chi- 
Square probability density) representing the colored object (figures 3 A, 3B and 4; column 4, 
lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 8, 
lines 10-38; column 12, lines 1-67). 
Claim 8: 

The claim 8 encompasses the same scope of invention as that of claim 7 except additional 
claimed limitation of the colored object not being tracked if the mean saturation or the standard 
deviation of the saturation is less than predetermined levels. However, Bradski further discloses 
the claimed hmitation of the selected object not being tracked if the mean saturation or the 



Application/Control Number: 09/822,923 Pa 8 e 7 

Art Unit: 2672 

standard deviation of the saturation is less than predetermined levels (figures 3 A, 3B and 4; 
column 4, lines 2-57, column 10, lines 1-16, column 5, lines 1-63, and column 6, lines 22-44; 
column 8, lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 

10. Claim 9: 

The claim 9 encompasses the same scope of invention as that of claim 1 in a method 
form. The claim is rejected for the same reason as set forth in claim 1 . 
Claim 10: 

The claim 10 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of converting a pixel data array for the at least one image frame 
from a red-green-blue color space (RGB) data array to the HSV data array. However, Bradski 
further discloses the claimed limitation of converting a pixel data array for the at least one image 
frame from a red-green-blue color space (RGB) data array to the HSV data array (column 4, 

lines 1-57). 

Claim 11: 

The claim 1 1 encompasses the same scope of invention as that of claim 28 except 
additional claimed limitation of applying the pixel data from an entire frame to the pixel- 
classification look-up map wherein if the amount of the pixels associated with the object is 
greater than a predetermined amount, the calibration method restarts. However, Bradski further 
discloses the claimed limitation of applying the pixel data from an entire frame to the pixel- 
classification look-up map (column 4, lines 45-60) wherein if the amount of the pixels associated 
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with the object is greater than a predetermined amount, the calibration method restarts (figures 
3A, 3B and 4; column 4, lines 1-61; column 6, lines 5-57; column 10, lines 35-67). 

Claim 13: 

The claim 13 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of thresholding the HSV data array of pixels and disregarding pixel 
data for each of the pixels having a product of a saturation coordinate and a value coordinate 
below a predetermined threshold amount. However, Bradski further discloses the claimed 
limitation of thresholding the HSV data array of pixels and disregarding pixel data for each of 
the pixels having a product of a saturation coordinate and a value coordinate below a 
predetermined threshold amount (From the Bradski' s teaching that "Hue values are only 
accumulated if their corresponding S and V values are above respective S and V thresholds". . . , it 
can be seen that pixel data with the product of S and V values less than a threshold would be 
excluded from accumulation to the Hue values. See figures 3 A, 3B and 4; column 4, lines 1-57; 
column 5, lines 1-61; column 10, lines 35-67; column 13, lines 55-67; column 14, lines 1-59). 

Claim 14: 

The claim 14 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean hue and a standard deviation of a hue of the 
pixels in the colored object. However, Bradski further discloses the claimed limitation of 
calculating a mean hue and a standard deviation of a hue of the pixels (The mean hue and 
standard deviation of a hue values are inherently associated with hue probability density) in the 
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colored object (column 4, lines 2-57, column 10, lines 1-16, column 5, lines 3 1-63, and column 
6, lines 22-44; column 8, lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 
Claim 15: 

The claim 15 encompasses the same scope of invention as that of claim 14 except 
additional claimed limitation of restarting the calibration method if the mean hue or the standard 
deviation of the hue is less than predetermined levels. However, Bradski further discloses the 
claimed limitation of restarting the calibration method (restarting searching) if the mean hue or 
the standard deviation of the hue is less than predetermined levels (figures 3 A, 3B and 4; column 

4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 1-44; column 8, 
lines 10-38; column 10, lines 35-67; column 12, lines 1-67). 

Claim 16: 

The claim 16 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of calculating a mean saturation and a standard deviation of a 
saturation of the pixels in the selected object. However, Bradski further discloses the claimed 
limitation of calculating a mean saturation and a standard deviation of a saturation of the pixels 
in the selected object (figures 3 A, 3B and 4; column 4, lines 2-57, column 10, lines 1-16, column 

5, lines 31-63, and column 6, lines 22-44; column 12, lines 1-67). 

Claim 17: 

The claim 17 encompasses the same scope of invention as that of claim 16 except 
additional claimed limitation of restarting the calibration method if the mean saturation or the 
standard deviation of the saturation is less than predetermined levels. However, Bradski further 
discloses the claimed limitation of restarting the calibration method if the mean saturation or the 
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standard deviation of the saturation is less than predetermined levels (figures 3 A and 3B; column 
4, lines 2-57, column 10, lines 1-16, column 5, lines 31-63, and column 6, lines 22-44; column 
12, lines 1-67). 
Claim 18: 

The claim 18 encompasses the same scope of invention as that of claim 10 except 
additional claimed limitation of allowing the user to select the selected object. However, Bradski 
further discloses the claimed limitation of allowing the user to select the selected object (column 
4, lines 1-57; column 15, lines 1-67). 

Claim 21: 

The claim 21 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of a thresholding module to disregard pixel data for each of the 
pixels having a product of a saturation coordinate and a value coordinate below a predetermined 
threshold amount. However, Bradski further discloses the claimed limitation of a thresholding 
module to disregard pixel data for each of the pixels having a product of a saturation coordinate 
and a value coordinate below a predetermined threshold amount (figures 3 A, 3B, 4-5; column 4, 
lines 1-57; column 13, lines 55-67; column 14, lines 1-59). 

Claim 22: 

The claim 22 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the calibration window being smaller than the at least one image 
frame. However, Bradski further discloses the claimed hmitation of the calibration window being 
smaller than the at least one image frame (figures 4, 10A-10B and 14A; column 6, lines 1-67; 
column 15, lines 1-67). 
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Claim 23: 

The claim 23 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of each of the adjacent test windows having a same size as the 
calibration window. However, Bradski further discloses the claimed limitation of each of the 
adjacent test windows having a same size as the calibration window (figures 4, 10A-10B and 
14A; column 6, lines 1-67; figures 11-14, column 15, lines 1-67). 

Claim 24: 

The claim 24 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of each of the adjacent test windows having at least one pixel 
overlapping with the calibration window. However, Bradski further discloses the claimed 
limitation of each of the adjacent test windows having at least one pixel overlapping with the 
calibration window (figures 4, 10A-10B and 14A; column 6, lines 1-67; figures 1 1-14, column 
15, lines 1-67). 

Claim 25: 

The claim 25 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of the calibration window being smaller than the at least one image 
frame. However, Bradski further discloses the claimed hmitation of the calibration window being 
smaller than the at least one image frame (figures 4, 10A-10B, and 14A; column 6, lines 1-67; 
column 15, lines 1-67). 

Claim 26: 

The claim 26 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of each of the adjacent test windows having a same size as the 
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calibration window. However, Bradski further discloses the claimed limitation of each of the 
adjacent test windows having a same size as the calibration window (figures 4, 10A, 10B and 
14A; column 6, lines 1-67; figures 11-14, column 15, lines 1-67). 



The claim 27 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of each of the adjacent test windows having at least one pixel 
overlapping with the calibration window. However, Bradski further discloses the claimed 
limitation of each of the adjacent test windows (the moving search windows) having at least one 
pixel overlapping with the calibration window (figures 4, 10A-10B; column 6, lines 1-67; figures 
11-14, column 15, lines 1-67). 



The claim 28 encompasses the same scope of invention as that of claim 9 except 
additional claimed limitation of creating a pixel-classification look-up map for the HSV data 
array of pixels. However, Bradski further discloses the claimed limitation of creating a pixel- 
classification look-up map for the HSV data array of pixels (figures 4, 10A and 10B; column 4, 
lines 2-57; column 6, lines 5-35). 

Remarks 

1 1 . Applicant's arguments, filed 06/06/2003, paper number 3, have been fully considered but 
they are not deemed to be persuasive. 

12. Applicant argues in essence with respect to claim 1 and the newly added claims that: 



Claim 27: 



Claim 28: 
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"However, Bradski, does not disclose, teach, or suggest use of an automated calibration 



calibration system including. . . " 
This is not found persuasive for the reasons given below. 

The examiner asserts that the claims are unpatentable over Bradski in the cited reference. 
Bradski teaches all the claimed limitations set forth in the amendment. In response to the 
claimed limitation set forth in claim 1 , 

Bradski teaches an automated calibration system (figure 1; column 3, lines 34-55) to 

track a selected object through a series of frames of data (column 7, lines 55-65), 

comprising: 

(a) A display device (e.g., figure 1 1) to display at least one image frame received from 
an image input device, wherein the image frame includes a calibration window (column 
4, lines 2-60; column 1 5, lines 13-58; figures 4, 1 OA and 10B); 

(b) An image selection device to select, via the calibration window, the selected object in 
the at least one image frame (column 3, lines 59-67, column 4, lines 1-10, and column 7, 
lines 33-54; column 15, lines 13-58); 

(c) An image source device (video source 1 102 of figure 1 1) to provide a hue saturation 
value (HSV) data array of pixels forming the at least one image frame (column 4, lines 2- 
57; column 13, lines 45-52; column 15, lines 13-58; figures 3A and 3B); and 

(d) An analysis module (the flow diagrams of figures 5, 7 and 9; column 13, lines 55-67; 
column 14, lines 1-59) to determine analysis data for pixels (column 4, lines 2-57; figures 
4, 3 A and 3B) within the calibration window (column 5, lines 52-67; column 6, lines 1- 



system to track a selected object through a series of frames of data, the automated 
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44; column 15, lines 13-58; figures4, 10A and 10B), based on the HSV data array 
(column 4, lines 2-57; figures 3 A and 3B), and determine test analysis data (e.g., window 
sizing parameters) for a set of adjacent test windows (e.g., search windows or boxes or 
windows of FIG. 14A displaying images; column 5, lines 52-67; column 6, lines 1-44; 
column 15, lines 13-58), each of the adjacent test windows having a same shape as the 
calibration window (e.g., the entire video frame or a search window; column 15, lines 13- 
58), wherein tracking data, to track the selected object (e.g., the selected gestures), is 
selected from one of the calibration window and the adjacent test windows having a 
highest tracking probability (i.e., the tracking data such as window location parameters 
are determined/adjusted and a search window having the largest connected region of a 
probability distribution and the highest probability density is selected, column 6, lines 45- 
67; column 7, lines 55-67; column 8, lines 1-9; column 15, lines 13-58). 
Therefore, Bradski fulfills the claim 1 as currently drafted. 

Conclusion 

13 THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 

policy as set forth in 37 CFR 1 . 136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
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CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

14. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 308-6606 for regular 
communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 395-3900. 



jew 




June 19, 2003 MICHAEL RAZAVI 

SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



